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Summary rel 2009-No. 066 

New measures of English language 
proficiency and their relationship to 
performance on large-scale content 
assessments 



Using assessment results for 5th and 8th 
grade English language learner students 
in three Northeast and Island Region 
states, the report finds that the English 
language domains of reading and writ- 
ing (as measured by a proficiency as- 
sessment) are significant predictors of 
performance on reading, writing, and 
mathematics assessments and that the 
domains of reading and writing (literacy 
skills) are more closely associated with 
performance than are the English lan- 
guage domains of speaking and listening 
(oral skills). 

As the English language learner population 
grows throughout the Northeast and Islands 
Region, state departments of education are 
seeking assistance in creating comprehen- 
sive approaches to meeting English language 
learner students’ academic needs in both 
instruction and assessment. Driving educa- 
tor concerns is the fact that English language 
learner students consistently score lower on 
state assessments than students for whom 
English is their first language. In the context of 
the No Child Left Behind Act of 2001 (NCLB), 
states are seeking information to inform 
their efforts to reduce achievement gaps and 
to bring English language learner students, 



along with other traditionally underserved 
student subgroups, to proficiency on statewide 
assessments. 

In response to a request from New Hampshire, 
Rhode Island, and Vermont to explore how 
English language proficiency measures may be 
related to performance outcomes on content 
assessments, this report uses the results of two 
new large-scale assessments — the Assessing 
Comprehension and Communication in Eng- 
lish State-to-State for English Language Learn- 
ers (ACCESS for ELLs) English proficiency 
assessment and the New England Common 
Assessment Program (NECAP) — to address 
the following research question: 

How does performance in four language 
domains on an English language profi- 
ciency assessment predict English lan- 
guage learner students’ performance on a 
state content assessment after accounting 
for student and school characteristics? 

Based on findings from previous research, 
this report hypothesized that after controlling 
for individual student characteristics such 
as gender, poverty status, disability status, 
race/ethnicity, age for grade, and years in 
English language learner programs as well as 
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for school characteristics such as school size, 
school poverty, racial composition, English 
language learner student density, and geogra- 
phy, measures of academic English language 
proficiency would predict English language 
learner student outcomes on state content as- 
sessments. The report also hypothesized that 
measures of English language literacy (reading 
and writing) would be stronger predictors of 
content assessment outcomes than would mea- 
sures of English oral proficiency (listening and 
speaking).^ 

To test these hypotheses, multilevel regression 
models were fit to assessment score data for 
5th and 8th grade English language learner 
students in New Hampshire, Rhode Island, 
and Vermont. After controlling for student and 
school characteristics, English language profi- 
ciency scores (as measured by ACCESS) were 
indeed significant predictors of content assess- 
ment outcomes (as measured by the NECAP). 
The models also showed that after accounting 
for other covariates, ACCESS measures of Eng- 
lish literacy were significantly stronger predic- 
tors of NECAP outcomes than were ACCESS 
measures of oral proficiency. Specifically, this 
report finds that: 

• NECAP reading scores in both 5th and 8th 
grades were significantly and positively 
predicted by ACCESS reading, writing, 
and speaking scores after controlling for 
other ACCESS scores and student and 
school characteristics. Among the ACCESS 
domain scores the strongest predictor of 
NECAP reading outcomes was ACCESS 
reading scores, followed by ACCESS writ- 
ing and speaking scores. ACCESS domain 
scores explained 30 percent of the variance 
in NECAP reading scores in 5th grade and 



23 percent in 8th grade after controlling 
for student and school covariates. 

• NECAP writing scores in 5th grade were 
significantly and positively predicted by 
ACCESS reading and writing scores and 
in 8th grade by all four ACCESS domain 
scores after controlling for other ACCESS 
scores and student and school character- 
istics. ACCESS reading and writing scores 
were the strongest predictors of NECAP 
writing outcomes in 5th and 8th grades. 
ACCESS domain scores explained 28 per- 
cent of the variance in NECAP writing 
scores in 5th grade and 25 percent in 8th 
grade after controlling for other covariates. 

• Like NECAP reading and writing scores, 
NECAP mathematics scores in both 5th 
and 8th grades were positively and sig- 
nificantly predicted by ACCESS reading 
and writing scores after controlling for 
other ACCESS scores and student and 
school characteristics. Among the ACCESS 
domain scores ACCESS reading scores 
were the strongest predictor of NECAP 
mathematics outcomes for both 5th and 
8th grade English language learner stu- 
dents, followed by ACCESS writing scores. 
ACCESS domain scores explained 21 
percent of the variance in NECAP math- 
ematics scores in 5th grade and 14 percent 
in 8th grade. 

• ACCESS reading and writing scores were 
significant predictors of NECAP reading, 
writing, and mathematics scores in 5th 
and 8th grades. ACCESS speaking and 
listening scores were significant predictors 
of NECAP scores for only four outcomes: 
5th and 8th grade reading (speaking), 8th 
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grade writing (speaking and listening), 
and 5th grade mathematics (listening). 

In sum, ACCESS measures of English literacy 
skills (reading and writing scores) were signifi- 
cant predictors of NECAP reading and writ- 
ing outcomes in 5th and 8th grades. Notably, 

ACCESS reading and writing scores were also 

positive and significant predictors of NECAP 

mathematics scores. In addition, except for 8th 

grade writing, ACCESS reading and writing 

scores were significantly stronger predictors of 

NECAP outcomes than were ACCESS listening 

and speaking scores. This evidence supports 

the original hypothesis that ACCESS measures Note 

of English literacy skills are better predictors 

of NECAP content outcomes than are ACCESS 

measures of English oral skills (listening and 

speaking). Readers are cautioned, however, 

that the analyses and interpretations presented 

are correlational and therefore do not allow 

causal conclusions. 



In 5th and 8th grades, ACCESS scores ex- 
plained 14-30 percent of the variance in scores 
for all three NECAP content scores (reading, 
writing, and mathematics) after controlling 
for background student and school charac- 
teristics. The ACCESS scores explained more 
of the variance in 5th grade (from 21 percent 
of NECAP mathematics scores to 30 percent 
of NECAP reading scores) than in 8th grade 
(from 14 percent of NECAP mathematics 
scores to 25 percent of NECAP writing scores). 
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1. In this report “stronger” predictors are de- 
fined as those whose regression coefficients are 
larger than those of other noted predictors in 
the study’s regression models. A predictor is 
“significantly stronger” than another predic- 
tor when the difference between the regression 
coefficients is greater than zero at the p < 0.05 
level. 



